lee/renderdoc
1
0
Fork 0
mirror of https://github.com/baldurk/renderdoc.git synced 2026-05-05 01:20:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix unpackData bug when unpacking UNorm and SNorm data

Browse Source
This commit is contained in:
sindney
2020-06-16 16:26:45 +08:00
committed by Baldur Karlsson
parent df575a17b4
commit 8e267c01ba
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/python_api/examples/renderdoc/decode_mesh.rst
+2 -2
View File
@@ -115,10 +115,10 @@ For normalised formats - :py:attr:`~renderdoc.CompType.UNorm` and :py:attr:`~ren
# If the format needs post-processing such as normalisation, do that now
if fmt.compType == rd.CompType.UNorm:
divisor = float((1 << fmt.compByteWidth) - 1)
divisor = float((2 ** (fmt.compByteWidth * 8)) - 1)
value = tuple(float(i) / divisor for i in value)
elif fmt.compType == rd.CompType.SNorm:
maxNeg = -(1 << (fmt.compByteWidth - 1))
maxNeg = -float((2 ** (fmt.compByteWidth * 8)) - 1) / 2
divisor = float(-(maxNeg-1))
value = tuple((float(i) if (i == maxNeg) else (float(i) / divisor)) for i in value)