Previously it did not (contrary to what was implied by the manual),
which means that an image with URL `/etc/passwd` would leak an
encoded version of that file to HTML output with `--self-contained`
or `--embed-resources`, even if `--sandbox` was used.
Thanks to Samuel Mortenson for pointing out the issue.
This change introduces a reader for mdoc, a roff-derived semantic markup
language for manual pages. The two relevant contemporary implementations
of mdoc for manual pages are mandoc (https://mandoc.bsd.lv/), which
implements the language from scratch in C, and groff
(https://www.gnu.org/software/groff/), which implements it as roff macros.
mdoc has a lot of semantics specific to technical manuals that aren't
representable in Pandoc's AST. I've taken a cue from the mandoc HTML
output and many mdoc elements are encoded as Codes or Spans with classes
named for the mdoc macro that produced them.
Much like web browsers with HTML, mandoc attempts to produce best-effort
output given all kinds of weird and crappy mdoc input. Part of the
reason it's able to do this is it uses a very accommodating parse tree
and stateful output routines specialized to the output mode, and when it
encounters some macro it wasn't expecting, it can easily give up on
whatever it was outputting and output something else. I've encoded as
much flexibility as I reasonably could into the mdoc reader here, but I
don't know how to be as flexible as mandoc.
This branch has been developed almost exclusively against mandoc's
documentation and implementation of mdoc as a reference, and the
real-world manual pages tested against are those from the OpenBSD base
system. Of ~3500 manuals in mdoc format shipped with a fresh OpenBSD
install, 17 cause the mdoc reader to exit with a parse error. Any
further chasing of edge cases is deferred to future work.
Many of the tests in test/Tests/Readers/Mdoc.hs are derived directly
from mandoc's extensive regression tests.
[API change] Adds readMdoc to the public API
The existing lexRoff does some stuff I don't want to deal with in mdoc
just yet, like lexing tbl, and some stuff I won't do at all, like
handling macro and text string definitions and switching between modes.
Uses a typeclass with associated type families to reuse most of the
escaping code between Roff (i.e. man) and Mdoc.
Future work could improve on this so that more lexing code could be
shared between Man and Mdoc. Mdoc inherits Roff's surface syntax so
hypothetically it makes sense to lex it into tokens that make sense for
roff. But it happens that the Mdoc parser is much easier to build with
an Mdoc specific token stream. Some discussion in jgm/pandoc#10225 about
the rationale.
Adds a test for the roff \A escape, which I accidentally dropped support
for in an earlier iteration without anything complaining.
The plain writer behaved as a markdown variant with Ext_line_blocks
turned off, and so empty lines in a line block would get eliminated.
This is surprising, since if there's anything where the intent can be
preserved in plain text output it's empty lines.
It's still a bit surprising to have nbsps in plain text output, as in
the test, where the distinction doesn't really matter, but that'd be an
orthogonal change.
from the citationItem id. In this case we use the citationItemId
in the bibliography as well, overriding the referenceId in the itemData.
Closes#10366.
Closes#10308.
The old method (checking to see if toc hash had changed) is not
completely reliable; there are cases where an additional run is
needed anyway to get the correct "logical" page
numbers (especially when different pagination is used for
front matter). See #10308 for an example.
Create MetaInlines when possible, just as with markdown input.
MetaBlocks is still used when there are multiple paragraphs or
non-paragraph content.
This change also affects field lists.
Closes#7766.
Instead of having an initial pass where we collect reference
definitions, we create links with target `##SUBST##something`
or `##REF##something` or `##NOTE##something`, and resolve these
in a pass over the parsed AST.
This allows us to handle link references that are not at the
top level. Closes#10281.
If `smart` is not enabled, a command in the default template will
disable smartquote substitutions.
When `smart` is enabled, render curly apostrophes as straight
and escape straight apostrophes.
When `smart` is disabled, render curly apostrophes as curly
and don't escape straight apostrophes.
And similarly for quotes, em and en dashes.
This should give more idiomatic typst output, with fewer unnecessary
escapes.
Closes#10271.
Previously these were only converted to a `<style>` element
if the link was being served with a text/css mime type (not
e.g. text/plain). We now add a secondary check for `rel="stylesheet"`
to better handle such cases.
Closes#10261.
Headings in docx, even ones that do not have a visible number,
can have a numId, and in odd cases can even share a numId with
a list that continues after the header. In this case the list
numbering should be reset by the header.
To accomplish this, we add a Heading constructor to BodyPart and
include on it all the information list items have.
Closes#10258.
John MacFarlane