Commit Graph

133 Commits

Author SHA1 Message Date
baldurk f2c401a255 Check that all states are present before allowing any stepping 2020-02-18 19:18:51 +00:00
baldurk 69a82202e5 Remove redundant/tautological checks 2020-02-13 11:01:45 +00:00
Steve Karolewics 671add4035 Fix shader debugging and pipeline viewer with constant buffer arrays
With SM5.1 and D3D12, constant buffer arrays can be declared which
are treated as an array of resources on the shader side, and a simple
collection of descriptors on the D3D12 side. DXBCDebug::GlobalState
handles the bridge between these by storing each array resource as a
nested ShaderVariable struct. Accessing the data for instructions
traverses the structure similarly. The shader viewer handles nested
resources and displays them in the appropriate tree view. In the
pipeline viewer, descriptors for CB arrays now indicate the array
index and correctly handle buffer offsets for viewing.
2020-02-11 17:10:57 +00:00
baldurk fe30fa91fb Fix python binding consistency with wrong naming scheme 2020-02-06 19:10:43 +00:00
baldurk 95ef40fe7c Add builtin to SourceVariableMapping 2020-02-06 17:58:42 +00:00
baldurk 661ee35f30 Refactor ShaderDebugTrace to not return a single list of complete states
* The ShaderDebugTrace now only sets up the initial state of an opaque
  ShaderDebugger handle.
* This handle can then be passed to a new function - ContinueDebug - to
  iteratively return N more states. The number of states is implementation
  defined and may be a fixed number or it may run for a fixed time.
* The states themselves no longer contain a complete snapshot of all variables,
  but instead only the changed variables for that iteration. The changes are
  stored as before and after value to make it easier to step forwards and
  backwards (only the ShaderDebugState is needed to move forward or backwards,
  you don't have to search back for the last set value of a variable to 'undo' a
  change).
2020-02-06 17:58:42 +00:00
baldurk 410f9d86bd Remove register/HLSL handling from shader viewer, work agnostically
* The shader viewer shouldn't handle specifics of D3D bytecode, instead it now
  works generally with whatever the debug and source variables are.
* VarType::Unknown is used for D3D registers which are typeless.
2020-02-06 17:58:41 +00:00
baldurk fdb6617cb1 Pass back 'registers' for resource bindings in debug trace 2020-02-06 17:58:41 +00:00
baldurk 0e0ff8714c Add fake source-mapping information for constants in debug traces
* cbuffer and input registers no longer have any names elided into them, they
  are 'raw' registers. The source mapping provides the information of where they
  come from as best as possible.
* Since we're now using source-mapping for constants, fix some source mapping
  issues with arrays, nested structs/arrays, and matrix major-ness.
2020-02-06 17:58:41 +00:00
baldurk 66cebe0a4c Refactor shader debug trace to have single mutable variable array
* Moving away from 'registers' explicitly being stored, we now have a single
  variable array where the shader representation is responsible for allocating
  it.
* In DXBC we calculate the number of each type of mutable variable (temps,
  indexable temps, and outputs) and assign slots linearly.
* We also update some naming - high-level variables aren't "locals" since they
  could be globals too, and debugging variables aren't "registers" but simply
  the variables we use for debugging which may be more complex than float4 for
  other APIs.
* The local variable matching by string name is unnecessary for this change
  but will be more useful in future after further refactors where there isn't
  a single list of variables to index into at the upper level.
2020-02-06 17:58:41 +00:00
baldurk 01461db738 Store mapping from instruction to disassembly line in shader debug trace
* This avoids having the UI need to do a scan and expecting to find
  "instruction:" on the start of a line.
* We also move the callstack to be stored per-step, and not in the line info
  mapping.
2020-02-06 17:58:41 +00:00
baldurk 14c1af06a3 Make ShaderViewer::ensureLineScrolled more conservative 2020-02-06 17:58:41 +00:00
baldurk 796f5ab328 Make sure scintilla margin is updated when text is updated 2020-02-06 17:58:41 +00:00
baldurk 08c0b13aaa Add further checks for windows being deleted during long async callbacks
* This is not really a full solution but fixes the common cases at least.
2020-02-05 15:06:53 +00:00
baldurk baac06ca93 Remove redundant/dead code identified by PVS Studio 2020-01-21 18:28:56 +00:00
baldurk 2916c0f9f7 Update copyright years to 2020 2020-01-06 16:20:45 +00:00
baldurk 6653316a62 Protect against window being closed during async render thread work 2019-11-12 20:44:35 +00:00
baldurk a0bc429bef Handle a source line corresponding to multiple instructions
* This can happen with inlined functions
2019-08-20 15:33:12 +01:00
baldurk e4333291a1 Allow using shader processing tools for custom shaders
* This is primarily useful for HLSL on Vulkan, but could be used with any other
  combination. If multiple tools can perform the conversion, the highest
  priority one is used.
2019-05-22 17:54:10 +01:00
baldurk 55107a6144 Only display textual shader encodings for shader compile options 2019-05-22 16:08:31 +01:00
baldurk e48065c96b Replace use of std::pair with rdcpair wherever possible
* Only remaining uses in our code is when we're interacting with std::map where
  it uses std::pair internally
2019-05-17 16:32:56 +01:00
baldurk 4311b35038 Tidy up RDTreeView expansion handling 2019-04-05 13:22:33 +01:00
baldurk ac91a7e8b4 Delete FindResults scintilla 2019-03-01 13:26:23 +00:00
baldurk fbb6b23b23 Support advanced cbuffer layouts
* This includes 8/16/64-bit integers, 16-bit/64-bit floats, and scalar block
  packing
2019-02-07 15:23:06 +00:00
baldurk 2853029847 Don't make overly-general matches when looking for constants by name 2019-02-04 18:06:10 +00:00
baldurk 36770522d7 Fix shaderviewer shortcuts using QShortcut which doesn't work well 2019-02-04 18:06:10 +00:00
baldurk 86ff87863e Update copyright years to 2019 2019-02-01 18:32:13 +00:00
baldurk ec6406c8ff Check next step is in bounds for states before checking conditions 2018-12-18 17:38:49 +00:00
baldurk 13e7d1c134 Support display of YUV textures on D3D11, D3D12 and Vulkan
* Supported textures are decoded into standard format of YUVA, displayed
  visually with Y in green, U in blue, V in red.
* A new texture display mode 'YUVA decode' has been added which does a default
  full-range conversion from YUV to RGB.
* Custom shaders can be used to implement a custom decode matrix.
2018-12-14 20:14:07 +00:00
baldurk 615b743b8e Make shader debugging soft-blocking, pop up progress bar while working
* This prevents the user from accidentally triggering some other process while
  the debug is still on-going, if it's taking a while.
2018-10-08 17:24:00 +01:00
baldurk 73ec1e1d37 Don't assume text output byte buffer is null terminated 2018-09-25 10:39:59 +01:00
baldurk a05d63852e Fix label for shader processing tools in disassembly dropdown 2018-08-28 18:23:57 +01:00
baldurk babe36bf7e Preserve stdout/stderr from external shader tools and display to user 2018-08-15 13:40:21 +01:00
baldurk 3c7b420e59 Expand SPIR-V disassemblers to general shader processing tools
* Instead of just configuring SPIR-V disassemblers and picking only the first
  one when we need to edit SPIR-V, we allow setting up any shader processor that
  goes between two shader encodings.
* When editing, the default will still be to use embedded source, and then after
  that the first tool that goes from the native shader format to a text format,
  but the drop-down allows you to pick any of them.
* Similarly in the shader viewer you can configure the compilation options and
  method, to choose the compiler you want to use. Embedded command line
  parameters in the shader are automatically appended.
2018-08-09 16:56:53 +01:00
baldurk a81e4a2b45 Pass through and allow different source shader encodings when editing
* This means e.g. the D3D11 back-end can accept DXBC directly if the UI can
  provide it, or compile from HLSL as before.
* More importantly, the Vulkan back-end can take SPIR-V compiled from any
  source, or compile from GLSL as before as a fall-back.
2018-08-09 16:56:52 +01:00
baldurk 74528c4d15 Allow saving state internally by key as well as externally 2018-08-06 16:23:03 +01:00
baldurk edc5815463 Move expansion saving to RDTreeView so it can be used in more places. 2018-08-06 16:23:02 +01:00
Dzmitry Malyshau 9c9e61a95e Remove unused this in lambdas of ShaderViewer 2018-07-03 22:08:28 +01:00
baldurk 89637d8b35 32-bit compile fix 2018-06-22 21:36:06 +01:00
baldurk ec2806df06 Add context menu for manipulating watch panel 2018-06-22 19:28:34 +01:00
baldurk fc3e527181 If D3DCOMPILE_SKIP_OPTIMIZATION is set, prefer source-level debugging 2018-06-22 19:28:34 +01:00
baldurk 04f214c768 Add toolbar button to switch between HLSL and assembly debugging
* This hopefully makes the HLSL debug mode more obvious to people.
2018-06-22 19:28:34 +01:00
baldurk c5689827a9 Allow clicking anywhere in rX.xyzw to highlight register 2018-06-22 19:28:34 +01:00
baldurk c0317855f8 Show tooltips for known local variables and constants 2018-06-22 19:28:34 +01:00
baldurk 2a6a0f0f95 Support local variables in watch expressions 2018-06-22 19:28:34 +01:00
baldurk 1761f5ab36 Highlight variables/registers that have changed 2018-06-22 19:28:33 +01:00
baldurk 5b3a12cd0c Save and restore tree expansion state when repopulating locals widget 2018-06-22 19:28:33 +01:00
baldurk cb0df2c844 Combine together structs/arrays in HLSL locals panel 2018-06-22 19:28:33 +01:00
baldurk 880f529fda Change mapping representation to be gather-based per variable
* Instead of having an N:N mapping of parts of variables to parts of registers,
  instead we gather everything together under each variable and it has a list of
  registers that comprise it.
* Any gaps are represented as undefined register mappings, for components that
  aren't available in any register.
2018-06-22 19:28:33 +01:00
baldurk 8b5ab06da7 Rename variables panel to registers 2018-06-22 19:28:32 +01:00