Files
renderdoc/docs/HowTo/HowToDebugShader.aml
T
baldurk c38affcded Initial commit of existing code.
* All renderdoc code up to this point was written by me, history is available by request
2014-05-02 08:33:01 +01:00

157 lines
9.1 KiB
XML

<?xml version="1.0" encoding="utf-8"?>
<topic id="8a7568ca-6b2c-4873-b8eb-d1ad2aff9629" revisionNumber="1">
<developerConceptualDocument xmlns="http://ddue.schemas.microsoft.com/authoring/2003/5" xmlns:xlink="http://www.w3.org/1999/xlink">
<introduction>
<para>This page goes into detail about how to set up your captures for debugging
shaders, as well as how to debug a shader and what controls are available.</para>
</introduction>
<section address="DebugInfo">
<title>Including debug info in shaders</title>
<content>
<para>For the most part at least some debug information is included with shaders unless
it is being explicitly stripped out at some point. There is usually an option to also
include additional debug information - such as original source code in a high-level language.
The exact process varies by API, but for D3D11 the flag /Zi to fxc or the equivalent
flag to D3DCompile() will include additional debugging information, and /Qstrip_debug
and /Qstrip_reflection will remove reflection information that can be useful - such as
the names of variables in constant buffers.</para>
</content>
</section>
<section address="vertex">
<title>Debugging a vertex</title>
<content>
<para>Vertex debugging is invoked from the mesh viewer. With the mesh viewer open you can
select the input vertex you wish to debug.</para>
<para>When a vertex is selected in the mesh data for the vertex input it will be highlighted
along with the primitive it is part of in the mesh display, provided the display is in
vertex input mode.</para>
<para>Either right click and choose debug vertex
from the context menu, or click on the debug icon in the toolbar.</para>
<mediaLink>
<caption placement="after" lead="Vertex Debugging">Launching vertex debugging from the mesh viewer.</caption>
<image xlink:href="VertexDebug"/>
</mediaLink>
<para>From here the debugger will open with the vertex shader in the shader debugger.
The inputs are automatically filled in from the mesh data.</para>
<alert class="note">
<para>Geometry and tessellation shaders are not yet debuggable.</para>
</alert>
</content>
</section>
<section address="pixel">
<title>Debugging a Pixel</title>
<content>
<para>Pixel debugging is launched from the texture viewer. For more details on selecting
the pixel to debug see <link xlink:href="F1C1E449-55ED-46FE-BBFD-11D1A0511501" />.</para>
<para>When a given pixel is selected you can either choose the debug icon on the
toolbar, <mediaLinkInline><image xlink:href="wrench"/></mediaLinkInline>
or click the debug button underneath the pixel context. Each of these will
launch the shader viewer with the selected pixel being debugged. The
inputs to the pixel will be automatically filled in.</para>
<para>There are currently several things to note while pixel debugging:</para>
<list class="bullet">
<listItem><para>There is no pixel history at the moment, so the drawcall that you wish to debug
must be selected as the current even when you attempt to debug the pixel.</para></listItem>
<listItem><para>If a drawcall overdraws the same pixel several times then the results of debugging
may not be accurate - RenderDoc attempts to pick the fragment that last passed the depth test but this
isn't guaranteed and may not be perfect. There's also no way to choose a different fragment in the case
of blending being used, or sample-frequency rendering.</para></listItem>
</list>
</content>
</section>
<section address="compute">
<title>Debugging a Compute thread</title>
<content>
<para>To debug a compute thread simply go to the compute shader section of the pipeline state
viewer and enter the group and thread ID of the thread you would like to debug. This thread
will be debugged in isolation as with any other shader invocation.</para>
<para>This means there can be no synchronisation with any other compute thread running and
the debugging will run from start to finish as if no other thread had run.</para>
<alert class="caution">
<para>This feature is <legacyBold>highly</legacyBold> experimental and is provided with no guarantees
yet! It may work on simple shaders which is why it is available.</para>
</alert>
</content>
</section>
<section address="debugging">
<title>Debugging Controls</title>
<content>
<para>When debugging, at the moment the controls are fairly basic.</para>
<mediaLink>
<caption placement="after" lead="Shader controls">Controls for stepping through shaders.</caption>
<image xlink:href="ShaderControls"/>
</mediaLink>
<para>The toolbar at the top gives controls for the program flow through the
shader. <mediaLinkInline><image xlink:href="runfwd"/></mediaLinkInline> Run and
<mediaLinkInline><image xlink:href="runback"/></mediaLinkInline> Run Backward simply
run from the current position all the way
through to the end or start of the program respectively.</para>
<para>The keyboard shortcuts for these controls are <command>F5</command> and
<command>Shift-F5</command> respectively.</para>
<para>You can set a breakpoint by pressing F9 (this will also remove a breakpoint that is
already there). When running in each direction or to cursor (see below) if execution hits
a breakpoint it will stop.</para>
<para>The other controls allow for single stepping and limited running.
<mediaLinkInline><image xlink:href="stepnext"/></mediaLinkInline> Step forward
will execute the current instruction and continue to the next - this includes following
any flow control statements such as jumps, loops, etc.</para>
<para><mediaLinkInline><image xlink:href="stepprev"/></mediaLinkInline> Step backwards
will jump back to whichever instruction lead to the current instruction.
This does not necessarily mean the previous instruction in the program as it could be
the destination of a jump. Stepping forwards and stepping backwards will always reverse
each other. The shortcuts for these commands are <command>F10</command> and <command>Shift-F10</command></para>
<para>The final control is to
<mediaLinkInline><image xlink:href="runcursor"/></mediaLinkInline> Run to the cursor.
This will perform in a similar fashion
to the "Run" command, but when it reaches the line that the cursor highlights it will stop
and pause execution. It will also stop if it reaches the end of the shader.</para>
<alert class="note">
<para>The highlighted instruction at any given point indicates the <legacyItalic>next</legacyItalic>
instruction to be executed - not the instruction that was just executed.</para>
</alert>
<para>There is also a toggle available to control the 'default' interpretation of temporary
register values - float or int. Since registers are typeless typically they are interpreted as
float values, but with this toggle you can toggle them to be interpreted as integers.</para>
</content>
</section>
<section address="debugging">
<title>Debugging Displays</title>
<content>
<para>Currently there is only a very basic display when debugging shaders.</para>
<para>There are two windows that display different types of registers. The constants window
will display input and constant buffer registers that are immutable throughout execution.
This will also list registers for resources and samplers (with basic format information).</para>
<mediaLink>
<caption placement="after" lead="Constants window">Constant, input and resource registers.</caption>
<image xlink:href="ShaderConsts"/>
</mediaLink>
<para>The other window will contain variable/mutable registers. These contain temporaries that
are typically allocated up front and will update as you step through the execution of the shader.
This window also contains the output registers.</para>
<mediaLink>
<caption placement="after" lead="Variable window">Variable registers - temporaries and outputs.</caption>
<image xlink:href="ShaderRegs"/>
</mediaLink>
<para>The final window is initially empty but can be filled out as needed. This shows custom watch
expressions and their values. Here you can write any expression involving an input, temporary or
output register along with a swizzle and typecast.</para>
<para>Swizzles follow the standard hlsl rules - .[xyzw] or .[rgba] in any permutation or repetition
will show those channels.</para>
<para>The custom typecast can be any of ,x ,i ,d ,f ,u to display the register as hex, signed integer,
double, float or unsigned respectively.</para>
<mediaLink>
<caption placement="after" lead="Watch window">Watch window - custom register expressions evaluated.</caption>
<image xlink:href="ShaderWatch"/>
</mediaLink>
</content>
</section>
</developerConceptualDocument>
</topic>