A character outside the Basic Multilingual Plane is written as two
consecutive \u control words holding the halves of a surrogate pair, as
in \uc1\u-10187 ?\u-9157 ?. Each half was converted on its own, and
since Text cannot hold an unpaired surrogate it became U+FFFD, so 𝐻 came
out as two replacement characters.
Hold a high surrogate back until we know whether the low surrogate
completing it follows, and combine the two. An unpaired surrogate
denotes no character, so it is now dropped rather than replaced; any text
other than the next \u ends the wait, so halves separated by text are not
combined.
Closes#11920.
Update tests.
Reason: it turns out that the native output generated by
pretty-simple isn't always readable by the native reader.
According to https://github.com/cdepillabout/pretty-simple/issues/99
it is not a design goal of the library that the rendered values
be readable using 'read'. This makes it unsuitable for our
purposes.
pretty-show is a bit slower and it uses 4-space indents
(non-configurable), but it doesn't have this serious drawback.
Previously we used our own homespun formatting. But this
produces over-long lines that aren't ideal for diffs in tests.
Easier to use something off-the-shelf and standard.
Closes#7580.
Performance is slower by about a factor of 10, but this isn't
really a problem because native isn't suitable as a serialization
format. (For serialization you should use json, because the reader
is so much faster than native.)
John MacFarlane