With SM5.1 and D3D12, constant buffer arrays can be declared which
are treated as an array of resources on the shader side, and a simple
collection of descriptors on the D3D12 side. DXBCDebug::GlobalState
handles the bridge between these by storing each array resource as a
nested ShaderVariable struct. Accessing the data for instructions
traverses the structure similarly. The shader viewer handles nested
resources and displays them in the appropriate tree view. In the
pipeline viewer, descriptors for CB arrays now indicate the array
index and correctly handle buffer offsets for viewing.
* The ShaderDebugTrace now only sets up the initial state of an opaque
ShaderDebugger handle.
* This handle can then be passed to a new function - ContinueDebug - to
iteratively return N more states. The number of states is implementation
defined and may be a fixed number or it may run for a fixed time.
* The states themselves no longer contain a complete snapshot of all variables,
but instead only the changed variables for that iteration. The changes are
stored as before and after value to make it easier to step forwards and
backwards (only the ShaderDebugState is needed to move forward or backwards,
you don't have to search back for the last set value of a variable to 'undo' a
change).
* The shader viewer shouldn't handle specifics of D3D bytecode, instead it now
works generally with whatever the debug and source variables are.
* VarType::Unknown is used for D3D registers which are typeless.
* cbuffer and input registers no longer have any names elided into them, they
are 'raw' registers. The source mapping provides the information of where they
come from as best as possible.
* Since we're now using source-mapping for constants, fix some source mapping
issues with arrays, nested structs/arrays, and matrix major-ness.
* Moving away from 'registers' explicitly being stored, we now have a single
variable array where the shader representation is responsible for allocating
it.
* In DXBC we calculate the number of each type of mutable variable (temps,
indexable temps, and outputs) and assign slots linearly.
* We also update some naming - high-level variables aren't "locals" since they
could be globals too, and debugging variables aren't "registers" but simply
the variables we use for debugging which may be more complex than float4 for
other APIs.
* The local variable matching by string name is unnecessary for this change
but will be more useful in future after further refactors where there isn't
a single list of variables to index into at the upper level.
* This avoids having the UI need to do a scan and expecting to find
"instruction:" on the start of a line.
* We also move the callstack to be stored per-step, and not in the line info
mapping.
* It's required for replay applications to call InitialiseReplay() before doing
any work, and to call ShutdownReplay() once they're finished.
* This lets us do more heavyweight shutdown work before global destructors are
being invoked and the RenderDoc instance is being destroyed.
* Anything that needs to be shut down during capture still has to happen in
RenderDoc::~RenderDoc since we obviously can't get the application to call a
shutdown function in that situation.
* It identifies some potential overflows which we know likely won't happen, but
the fixes are harmless.
* Some issues are definitely bad, though not actual bugs.
* This is needed so that the functional tests can elevate and run renderdoccmd
to register the vulkan layer, if needed.
* At the same time remove the old spammy message and ignore flag - this dates
back to before the UI existed, and that should be the way users run RenderDoc
generally and it has a good UI for walking through layer registration if
needed.
* The command is always available, but will only show up in help if attention is
needed.
* Also fix registering installs on shared drives.
* Mostly moving includes from common headers to cpp where possible, and removing
includes of the whole thing where only enums or rdcstr etc are needed.
Steve Karolewics