Commit Graph

46 Commits

Author SHA1 Message Date
baldurk 83d77ea543 Improve formatting of Iter_Test output 2020-02-18 19:18:51 +00:00
baldurk 06570ea7ad Remove backbuffer comparisons from test suite, simplify default triangle
* It's not particularly scalable and can be brittle to driver changes, and we
  can use targeted specific pixel tests to check what we really want - to see if
  the output has rendered correctly.
* Overlay tests still check files directly - this is a future refactor to
  remove.
2020-02-11 17:11:33 +00:00
baldurk 76a88cc7a3 Add test of D3D12 render passes 2020-02-11 17:11:32 +00:00
baldurk e012c82168 Update tests to use new interfaces 2020-02-06 17:58:42 +00:00
baldurk b38cbc1b0c Add explicit functions to initialise and shutdown replay. Closes #1685
* It's required for replay applications to call InitialiseReplay() before doing
  any work, and to call ShutdownReplay() once they're finished.
* This lets us do more heavyweight shutdown work before global destructors are
  being invoked and the RenderDoc instance is being destroyed.
* Anything that needs to be shut down during capture still has to happen in
  RenderDoc::~RenderDoc since we obviously can't get the application to call a
  shutdown function in that situation.
2020-01-30 13:39:15 +00:00
baldurk db0779e09b Add test for mesh view - vertex picking and mesh rendering 2020-01-21 18:28:55 +00:00
thisisjimmyfb 0734776a49 added test demo and script for opengl backdoor context 2020-01-20 13:41:18 +00:00
baldurk 42841e23be Expose vulkan layer registration in renderdoccmd on win32. Closes #1690
* This is needed so that the functional tests can elevate and run renderdoccmd
  to register the vulkan layer, if needed.
* At the same time remove the old spammy message and ignore flag - this dates
  back to before the UI existed, and that should be the way users run RenderDoc
  generally and it has a good UI for walking through layer registration if
  needed.
* The command is always available, but will only show up in help if attention is
  needed.
* Also fix registering installs on shared drives.
2020-01-20 11:46:48 +00:00
baldurk 2418a8faef Add a per-test duration log when running tests 2020-01-15 11:24:41 +00:00
baldurk 0f26e3cb6e Allow images to be unsupported 2019-12-13 11:46:28 +00:00
baldurk 279a2ec69d Add texture zoo tests
* These tests ensure that texture rendering works correctly for all different
  types of texture types, and for all formats, across different APIs, including
  across a remote-proxy connection.
2019-11-26 17:38:27 +00:00
baldurk db563bb0bf Refactor public interface around handling of textures
* Subresource handling is more consistent - we pass around a struct now that
  contains the array slice, mip level, and sample. We remove the concept of
  'MSAA textures count samples as extra slices within the real slices' and
  internalise that completely. This also means we have a consistent set
  everywhere that we need to refer to a subresource.
* Functions that used to be in the ReplayOutput and use a couple of implicit
  parameters from the texture viewer configuration are now in the
  ReplayController and take them explicitly. This includes GetMinMax,
  GetHistogram, and PickPixel.
* Since these functions aren't ReplayOutput relative, if you want to decode the
  custom shader texture or the overlay texture you need to pass that ID
  directly.
2019-11-26 17:38:25 +00:00
baldurk 1530bc6499 Let tests know if they're being run in debug mode
* This will let us do extra logging or dumping data we don't want to do on a
  real run
2019-10-24 18:02:54 +01:00
baldurk 1eae20cad3 Add test of exporting/importing capture with a large number of buffers 2019-10-24 18:02:54 +01:00
baldurk dff2583db3 Add tests for specific parameters 2019-09-13 17:33:48 +01:00
baldurk a2df78369c Add test of denormal flushing to D3D11 shader debug zoo 2019-09-13 17:33:48 +01:00
baldurk d71b15115c Add additional proteciton & testing of unbound images in vulkan 2019-09-13 17:33:48 +01:00
baldurk 9627332667 Add a utility function to tests to pick a pixel and check its value 2019-09-05 20:33:25 +01:00
baldurk 4a4884129b Allow overriding replay options in test cases 2019-09-05 11:48:47 +01:00
baldurk 096e5af08b Fix graphics tests to pass default replay options to OpenCapture 2019-08-29 10:02:29 +01:00
baldurk dbd8a99a61 Allow a custom epsilon for value comparisons in tests 2019-07-25 12:25:40 +01:00
baldurk 587d562fa3 Fix NaN and INF value comparisons in tests 2019-06-03 14:16:43 +01:00
baldurk d86d270f3e Fix float value comparisons, only consider values equal if both are NaN 2019-05-29 10:49:59 +01:00
baldurk 4b1f527eaa Clean up D3D11 shader debugging test for easier processing, auto-test it 2019-05-27 13:58:08 +01:00
baldurk 7ab3d943b0 Add helper function to look up ResourceDescription for a ResourceId 2019-05-27 11:48:06 +01:00
baldurk 6e2e2abb22 Fix check for strip restart, only apply if an index buffer is active 2019-05-24 17:12:55 +01:00
baldurk 2d370ac4ae Add functions to check vertex input data as well as vertex output data 2019-05-24 17:12:54 +01:00
baldurk ce8fcc1a16 Fix UNorm/SNorm post-processing on mesh data decode 2019-05-24 13:31:48 +01:00
baldurk f6a2bd345c Fix PostVS alignment calculation in test mesh fetch 2019-05-24 13:31:24 +01:00
baldurk 786d6c84af Link-ify the git commit in test results 2019-05-24 11:52:54 +01:00
baldurk 4cec51ff6a Use demo project name to determine test availability
* This is better than testing by platform/platform version, because we can check
  specifically for extensions and avoid running a test we know will fail because
  extensions aren't supported.
2019-05-23 14:03:47 +01:00
baldurk efb8788c52 Identify the demos binary in scripts, and allow manual configuration 2019-05-23 11:07:37 +01:00
baldurk 5c28bd0a31 Add a parameter to allow saving RGB PNGs 2019-05-06 15:57:02 +01:00
baldurk 1bef46bfee Add a debugger friendly option to the python test framework
* This suppresses the use of a global try/except to catch exceptions then log &
  continue. Instead it runs the test and lets the debugger catch any exceptions
  that happen.
2019-05-03 11:13:52 +01:00
baldurk b680d74efd In python tests allow comparing RGB and RGBA PNGs
* We force the missing alpha channel to full 255 on all pixels.
2019-05-03 11:00:59 +01:00
baldurk 5484bcbf1a Don't error if a non-existant file is passed via TestFailureException 2019-02-15 18:17:20 +00:00
baldurk 7ef8ed0a7a Drop the requirement for PIL in the tests, use pypng drop-in replacement
* This means we no longer need to import PIL from somewhere else if we run via
  renderdoccmd test functional
2019-02-15 18:17:06 +00:00
baldurk fbb6b23b23 Support advanced cbuffer layouts
* This includes 8/16/64-bit integers, 16-bit/64-bit floats, and scalar block
  packing
2019-02-07 15:23:06 +00:00
baldurk 134cdfd09b Add tests to CBuffer Zoos to test alternate uniform sources on APIs
* Vulkan: Specialization constants
* OpenGL: Bare uniforms (not in a UBO)
* D3D12: Root signature constants
2019-02-07 15:23:06 +00:00
baldurk 351c75e006 Fix some cases handling arrays of matrices
* We also pick the output pixel in the CBuffer_Zoo tests to ensure the API
  agrees with our interpretation of the data.
* Follow-up commit will tidy D3D cbuffer code that needs it.
2019-02-07 15:23:05 +00:00
baldurk 0cee61558a Mention the index when some PostVS data doesn't match ref data in test 2019-01-22 16:55:39 +00:00
baldurk 2237c241ff Fix incorrectly named methods in ResourceFormat
* We try to maintain at least an internally consistent naming scheme for the
  python/public interface, even if it doesn't match python naming schemes.
2019-01-03 12:22:29 +00:00
baldurk d777f6bf9d Allow sourcing data from disk in demos program 2018-12-12 13:55:17 +00:00
baldurk 1f0192ba3e Fix python not calling bgraOrder function right (it used to be a flag) 2018-12-12 13:55:16 +00:00
baldurk 5b11cfea0a If recompressed capture doesn't match, include captures in artifacts 2018-12-06 16:36:14 +00:00
baldurk a9151a8c0b Add testing framework useful for automated testing
* Built entirely independently of the main renderdoc build (for now).
* Contains python scripts & framework for running tests on a renderdoc build,
  which will be run nightly.
2018-11-29 14:47:20 +00:00