* The ShaderDebugTrace now only sets up the initial state of an opaque
ShaderDebugger handle.
* This handle can then be passed to a new function - ContinueDebug - to
iteratively return N more states. The number of states is implementation
defined and may be a fixed number or it may run for a fixed time.
* The states themselves no longer contain a complete snapshot of all variables,
but instead only the changed variables for that iteration. The changes are
stored as before and after value to make it easier to step forwards and
backwards (only the ShaderDebugState is needed to move forward or backwards,
you don't have to search back for the last set value of a variable to 'undo' a
change).
* The shader viewer shouldn't handle specifics of D3D bytecode, instead it now
works generally with whatever the debug and source variables are.
* VarType::Unknown is used for D3D registers which are typeless.
* cbuffer and input registers no longer have any names elided into them, they
are 'raw' registers. The source mapping provides the information of where they
come from as best as possible.
* Since we're now using source-mapping for constants, fix some source mapping
issues with arrays, nested structs/arrays, and matrix major-ness.
* Moving away from 'registers' explicitly being stored, we now have a single
variable array where the shader representation is responsible for allocating
it.
* In DXBC we calculate the number of each type of mutable variable (temps,
indexable temps, and outputs) and assign slots linearly.
* We also update some naming - high-level variables aren't "locals" since they
could be globals too, and debugging variables aren't "registers" but simply
the variables we use for debugging which may be more complex than float4 for
other APIs.
* The local variable matching by string name is unnecessary for this change
but will be more useful in future after further refactors where there isn't
a single list of variables to index into at the upper level.
* This avoids having the UI need to do a scan and expecting to find
"instruction:" on the start of a line.
* We also move the callstack to be stored per-step, and not in the line info
mapping.
* It's required for replay applications to call InitialiseReplay() before doing
any work, and to call ShutdownReplay() once they're finished.
* This lets us do more heavyweight shutdown work before global destructors are
being invoked and the RenderDoc instance is being destroyed.
* Anything that needs to be shut down during capture still has to happen in
RenderDoc::~RenderDoc since we obviously can't get the application to call a
shutdown function in that situation.
* This is needed so that the functional tests can elevate and run renderdoccmd
to register the vulkan layer, if needed.
* At the same time remove the old spammy message and ignore flag - this dates
back to before the UI existed, and that should be the way users run RenderDoc
generally and it has a good UI for walking through layer registration if
needed.
* The command is always available, but will only show up in help if attention is
needed.
* Also fix registering installs on shared drives.
* Mostly moving includes from common headers to cpp where possible, and removing
includes of the whole thing where only enums or rdcstr etc are needed.
This fixes an infinite when attempting to call `insert(0, NULL, 0)`
on an empty `rdcarray`.
The new behaviour also triggers a copy when the inserted interval
overlaps the `rdcarray`'s *allocated* memory (rather than just the
currently used prefixed).
Change-Id: I075827d50a24d0560ca391224820334e467a41f5
* Subresource handling is more consistent - we pass around a struct now that
contains the array slice, mip level, and sample. We remove the concept of
'MSAA textures count samples as extra slices within the real slices' and
internalise that completely. This also means we have a consistent set
everywhere that we need to refer to a subresource.
* Functions that used to be in the ReplayOutput and use a couple of implicit
parameters from the texture viewer configuration are now in the
ReplayController and take them explicitly. This includes GetMinMax,
GetHistogram, and PickPixel.
* Since these functions aren't ReplayOutput relative, if you want to decode the
custom shader texture or the overlay texture you need to pass that ID
directly.
* Previously we had "Frame X" and "Start of Frame" hardcoded in the event
browser, and the end of frame was in many cases assumed to be a present call.
However with the in-application API this is not necessarily true.
* Presents are now serialised separately in all APIs and displayed wherever they
happen in the frame, and if there is no present at the end of the frame an
"End of Capture" marker is inserted. Similarly API-defined captures are not
given a potentially misleading frame number.
* In particular, if there is a mismatched layer registered under e.g. /etc then
we always need to elevate, even if the user wants their layer to be fixed and
registered user-local.
baldurk