This change introduces a reader for mdoc, a roff-derived semantic markup
language for manual pages. The two relevant contemporary implementations
of mdoc for manual pages are mandoc (https://mandoc.bsd.lv/), which
implements the language from scratch in C, and groff
(https://www.gnu.org/software/groff/), which implements it as roff macros.
mdoc has a lot of semantics specific to technical manuals that aren't
representable in Pandoc's AST. I've taken a cue from the mandoc HTML
output and many mdoc elements are encoded as Codes or Spans with classes
named for the mdoc macro that produced them.
Much like web browsers with HTML, mandoc attempts to produce best-effort
output given all kinds of weird and crappy mdoc input. Part of the
reason it's able to do this is it uses a very accommodating parse tree
and stateful output routines specialized to the output mode, and when it
encounters some macro it wasn't expecting, it can easily give up on
whatever it was outputting and output something else. I've encoded as
much flexibility as I reasonably could into the mdoc reader here, but I
don't know how to be as flexible as mandoc.
This branch has been developed almost exclusively against mandoc's
documentation and implementation of mdoc as a reference, and the
real-world manual pages tested against are those from the OpenBSD base
system. Of ~3500 manuals in mdoc format shipped with a fresh OpenBSD
install, 17 cause the mdoc reader to exit with a parse error. Any
further chasing of edge cases is deferred to future work.
Many of the tests in test/Tests/Readers/Mdoc.hs are derived directly
from mandoc's extensive regression tests.
[API change] Adds readMdoc to the public API
The existing lexRoff does some stuff I don't want to deal with in mdoc
just yet, like lexing tbl, and some stuff I won't do at all, like
handling macro and text string definitions and switching between modes.
Uses a typeclass with associated type families to reuse most of the
escaping code between Roff (i.e. man) and Mdoc.
Future work could improve on this so that more lexing code could be
shared between Man and Mdoc. Mdoc inherits Roff's surface syntax so
hypothetically it makes sense to lex it into tokens that make sense for
roff. But it happens that the Mdoc parser is much easier to build with
an Mdoc specific token stream. Some discussion in jgm/pandoc#10225 about
the rationale.
Adds a test for the roff \A escape, which I accidentally dropped support
for in an earlier iteration without anything complaining.
The plain writer behaved as a markdown variant with Ext_line_blocks
turned off, and so empty lines in a line block would get eliminated.
This is surprising, since if there's anything where the intent can be
preserved in plain text output it's empty lines.
It's still a bit surprising to have nbsps in plain text output, as in
the test, where the distinction doesn't really matter, but that'd be an
orthogonal change.
Instead of having an initial pass where we collect reference
definitions, we create links with target `##SUBST##something`
or `##REF##something` or `##NOTE##something`, and resolve these
in a pass over the parsed AST.
This allows us to handle link references that are not at the
top level. Closes#10281.
If `smart` is not enabled, a command in the default template will
disable smartquote substitutions.
When `smart` is enabled, render curly apostrophes as straight
and escape straight apostrophes.
When `smart` is disabled, render curly apostrophes as curly
and don't escape straight apostrophes.
And similarly for quotes, em and en dashes.
This should give more idiomatic typst output, with fewer unnecessary
escapes.
Closes#10271.
+ Remove section numbering code from common.latex
+ Add section numbering to default.latex
+ Add logic for numbering sections in default.beamer. I moved the template setting
code to where other beamer templates are set. This makes the section-titles and
numbersections variables independent.
This should make --number-sections work with beamer.
In these cases we emit an empty comment to fix the point from
which indentation is measured; otherwise the block quote is not
parsed as a block quote.
This affects list items and admonitions.
Cloess #10236.
In commit a26ec96d89 we added an
empty `\item[]` to the beginning of a list that occurs first
in a definition list, to avoid having one item on the line with
the label.
This gave bad results in some cases (#10241) and there is a more
idiomatic solution anyway: using `\hfill`.
Closes#10241.
+ Split out common parts of latex template into partials: common.latex,
fonts.latex, font-settings.latex, passoptions.latex, hypersetup.latex,
after-header-includes.latex.
+ Split out old latex template into default.latex and default.beamer.
+ Make default.beamer the default template for beamer.
- Fix alignment of list items corresponding to cells.
- Don't enclose the list table in a `.. table::`; this leads to
doubled captions.
Closes#10226.
Closes#10227.
Modified test output for #4564.
The new commonmark-pandoc version automatically adds the
attribute `wrapper="1"` on all Divs and Spans that
are introduced just as containers for attributes that belong
properly to their contents. So we don't need to add the
attribute here.
This gives much better results in some cases. Previously the
wrapper attribute was being added even for explicit Divs and
Spans in djot, but it is not needed in these cases.
This is like `makeSections` but has an additional parameter
specifying number offsets, for use with the `--number-offset` option.
Use `makeSectionsWithOffsets` in HTML writer instead of ad hoc
and inefficient number-adjusting code.
Clarify MANUAL.txt: the `--number-offset` option should only
directly affect numbering of the first section heading in
a document; subsequent headings will increment normally.
Fix test output for #5071 to reflect this.
+ Add command line options `--table-caption-position` and
`--figure-caption-position`. These allow the user to specify whether
to put captions above or below tables and figures, respectively.
The following output formats are supported: HTML (and related such
as EPUB), LaTeX (and Beamer), Docx, ODT/OpenDocument, Typst.
+ Text.Pandoc.Options: add `CaptionPosition` and new
`WriterOptions` fields `writerFigureCaptionPosition` and
`writerTableCaptionPosition` [API change].
+ Text.Pandoc.Opt: add `Opt` fields `optFigureCaptionPosition` and
`optTableCaptionPosition` [API change].
+ Docx writer: make table/figure rendering sensitive to caption
position settings.
+ OpenDocument writer: make table/figure rendering sensitive to
caption position settings.
+ Typst writer/template: implement figure caption positions by
triggering a show rule in the default template, which determines caption
positions for figures and tables globally.
+ LaTeX writer: make table/figure rendering sensitive to caption
position settings. Closes#5116.
+ HTML writer/template: make `<figcaption>` placement sensitive to caption
position settings. For tables, `<caption>` must be the first element,
and positioning is determined by CSS, for here we set a variable
which the default template is sensitive to.
Multiline rows occur only when the *first* cell is empty; we were
previously treating lines with *any* empty cell as row continuations.
Closes#10093.
In addition, we no longer wrap multiline cells in Para if they
can be represented as Plain. This is consistent with docutils
behavior.
We used to normalize so that the base level is always 1, but
asciidoc no longer seems to care about that, and the behavior
creates difficulties when we are converting fragments.
Closes#10062.
In #9275 we made pandoc strip off locator labels (e.g. `p.`)
for natbib and biblatex output. In fact, this is only desirable
for biblatex. natbib needs the locators to be preserved.
Closes#10057.
include `header`. This change causes pandoc to omit the middle
header parameter when `header` is not set, rather than
emitting `""`. The parameter is optional and man will use
a default based on the section if it is not specified.
Closes#9943.
Previously we relied on graphicx internals and made global
changes to Gin to force images to be resized if they exceed
textwidth. This approach is brittle and caused problems
with `\includesvg` (see #9660).
The new approach uses a new macro `\pandocbounded` that is
now defined in the LaTeX template. (Thanks here to Falk Hanisch in
https://github.com/mrpiggi/svg/issues/60.)
The LaTeX writer has been changed to enclose `\includegraphics`
and `\includesvg` commands in this macro when they don't explicitly
specify a width or height.
In addition, the writer now adds `keepaspectratio` to the
`\includegraphics` or `\includesvg` options if `height` is specified
without width, or vice versa. Previously, this was set in the preamble
as a global option.
Compatibility issues:
- If custom templates are used with the new LaTeX writer, they will have
to be updated to include the new `\pandocbounded` macro, or an error
will be raised because of the undefined macro.
- Documents that specify explicit dimensions for an image may render
differently, if the dimensions are greater than the line width or
page height. Previously pandoc would shrink these images to fit,
but the new behavior takes the specified dimensions literally.
In addition, pandoc previously always enforced `keepaspectratio`,
even when width and height were both specified, so images with
width and height specified that do not conform to their intrinsic
aspect ratio will appear differently.
Closes#9660.
John MacFarlane