The contents of footnotes, endnotes and comments were read with the
namespace prefixes declared in word/document.xml rather than in their
own parts. If word/footnotes.xml, word/endnotes.xml or word/comments.xml
declared a prefix that document.xml did not, such as `r` or `m`, the
elements using it were dropped: a hyperlink in a footnote lost its text,
and an equation in an endnote or comment disappeared. They are now read
with the namespace maps that archiveToNotes and archiveToComments
already compute.
Closes#11991.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
We try to identify which class is a language name and emit
that as the gfm style info string.
If none of the classes match a recognized language, before
3.10.1 we emitted the first class; fater 3.10.1, we emit
nothing. This commit reverts to the pre-3.10.1 behavior.
Closes#11987.
`w:w` is optional on `w:gridCol`, but elemToTblGrid skipped the grid
columns that had none. The table then had fewer columns than its rows
had cells, and the extra cells were dropped. Every grid column is now
kept. If any of them has no (readable) width, all columns get the
default width, as for a table without `w:tblGrid`, since writers don't
handle a mix of specified and default widths well.
Closes#11989.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A `w:fldSimple` element had no case in `elemToParPart'`, so `mapD` dropped it
together with its result runs: "Figure <fldSimple>1</fldSimple> Caption"
read as "Figure Caption", and a simple REF field lost its text. It is now
read as a Field with its runs as children, like a complex field.
`stripCaptionLabel` relied on that drop to remove the number after an
English Figure/Table label; it now drops the number field explicitly, so
those captions read as before.
Closes#11944.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Previously the docx writer gave the second and subsequent paragraphs
of a list item numbering from a separate list with an invisible
bullet. This looked right, but semantically these paragraphs were
items of another list, which is what Word, screen readers, and
pandoc's own docx reader saw; the list was also split in three.
The writer now gives these paragraphs no numbering, just an
indentation that lines them up with the text of the list item
(as Word's "List Continue" styles do).
The docx reader now records the indentation of each list level
and tracks the open list levels; an unnumbered paragraph whose
left indentation matches the text indentation of an open level
is treated as a continuation of that level's current item,
rather than as a block quote. So multi-paragraph list items
now round-trip through docx.
Closes#9197.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A macro whose expansion contains itself after an unknown command,
e.g. `\def\small{\@setfontsize\small\@ixpt{10}}`, was expanded
again and again at the same position, each time in a fresh call
to doMacros, so the existing depth check never triggered and
pandoc hung. We now track consecutive expansions of the same
macro at the same position and raise PandocMacroLoop after 1000.
Closes#11088.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
We were consuming certain symbols after `\uNNNN` during tokenization.
This led to an *extra* character being consumed as the "fallback,"
so letters went missing after encoded unicode characters.
Closes#11981.
Rendering a note can add further notes to the state. Previously
these were lost. Now we keep rendering notes until no new ones
appear, as the Org writer already does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pandoc parsed YAML with the yaml package, which depends on the C
library libyaml. It now uses yamlet, a pure Haskell YAML 1.2 parser.
Values still go through the existing aeson instances.
Behavior changes:
- YAML errors show the file, the line and the column, with the
offending line. In metadata blocks the line is the line in the
document.
- `yes`, `no`, `on`, `off`, `y` and `n` are strings,
not booleans, as in YAML 1.2. This restores the behavior before
the switch from HsYAML.
- Duplicate keys in a mapping are an error. They were a warning.
- .5, .inf and .nan are numbers.
Text.Pandoc.Translations: readTranslations takes the path of the
file, for error messages [API change].
A table was written as a grid table only if it had spans or a footer,
or if `writerColumns >= 8 * numcols`. A wider table with block content
in a cell therefore fell back to raw HTML, even with grid_tables
enabled; with the default 72 columns that is any table of ten or more
columns.
The width check was added in ea619bfcb (2017, #2834) to fix grid
tables nested in grid tables. The old grid writer gave each column
`floor(columns * share) - 3` characters and rendered cell contents with
`writerColumns` reduced to the column width, so nested tables got
narrower at each level. Below about 8 characters per column those
widths reached zero or went negative and the layout broke.
The `gridTable` rewrite in 01257ef6e (pandoc 3.7) removed that failure
mode. A column is never made narrower than its content (or, when
wrapping, than its longest unbreakable word), and cells are rendered
with unchanged `opts`, so `writerColumns` no longer shrinks for nested
content. The RST and Muse writers already call the same `gridTable`
with no width check, and the Markdown writer already writes grid tables
wider than `columns` when a table has spans or a footer.
The check is therefore removed. This follows the preference stated in
7c2dd0359 for markdown tables over raw HTML. No existing test output
changes.
Closes#11945.
A change in 3.12 broke escaping of data URIs, so that
`--embed-resources` no longer worked properly with JavaScript
includes, breaking slide show formats.
Closes#11942.
Prevent math delimiters inside TeX brace groups and percent comments
from prematurely closing an equation:
* Generalize the existing inline `\text` group handling to arbitrary
brace groups, and support groups in display math too.
* Recognize unescaped `%` as starting a comment through the next newline.
Ignore any braces or math delimiters within it.
Developed with OpenAI Codex.
PR revised by John MacFarlane.
The command test extracted the xlink:href by piping pandoc's output
into grep, and the command tests run under cmd.exe on Windows, where
there is no grep and the single quotes around the pattern are not
stripped. Nothing matched, and the test failed.
Tests.Writers.ODT makes the same assertion by parsing the output and
comparing the xlink:href of every draw:image. Reading the output back
in would not do instead: fixRelativeLink strips a leading "../", so a
flat file that wrongly had one would still read back as "lalune.jpg".
in structured CSL YAML references. Thus,
family: de Gaulle
should not create a non-dropping particle "de"; "de Gaulle"
should be considered the integral family name.
This is a behavior change that could affect some bibliography
processing.
The commit 4be49d0ce9 is
also reverted (adding a section explaining the former behavior
to the manual).
Closes#11911.
The fixtures in test/odt/fodt are LibreOffice's own conversions of the
zipped fixtures in test/odt/odt, and they are compared against the very
same expectations, so that flat and zipped input are asserted to
converge on identical documents.
There are only four of them, because a flat file shares nearly all of
the reader with a zipped one: once readFODT has handed the root element
over, the converter is the very same code. Two of the four are there
for the two paths that a flat file alone reaches: image, whose data has
to be decoded from office:binary-data and typed from draw:mime-type
rather than found in the archive, and formula, whose MathML is read
from the draw:object itself rather than from a sub-document. The other
two guard against whitespace, which is peculiar to the format without
being a separate branch of the reader, since LibreOffice pretty-prints
a flat file and that indentation is in the XML the reader sees:
preformattedText is a CodeBlock, whose whitespace is significant and so
is compared exactly, and textMixedStyles is a run of nested emphasis,
where stray indentation would show up between the inlines.
`test/command/fodt-roundtrip.md` round-trips markdown through the flat
format, which also covers the writer, whose output has a timestamp in
it and so cannot be compared byte for byte.
Together with previous commits, closes#4010.
The text styles were kept in a map keyed on the set of style
attributes, with the generated style name as part of the value, and
that name was used to order the style definitions in
`office:automatic-styles`. Sorting names as text puts `T10` between
`T1` and `T2`, so a document with ten or more distinct text styles
emitted them in a confusing order.
Store the style's number rather than its name, which makes the
existing sort numeric, and derive the name from it.
Co-Authored-By: Claude <noreply@anthropic.com>
Every blockquote, every explicitly aligned table cell, and every
direction-adjusted style got a fresh `Pn` automatic style, even when
an identical one already existed. A document with many of them paid
for a style definition each time, and the style list that gets walked
when emitting `office:automatic-styles` grew with it.
Key the styles on their attributes and paragraph properties and reuse
a match. The direction cache added for RTL support is subsumed by
this one and goes away.
The test output changes only by dropping duplicate definitions:
`writer.opendocument` had three identical `Quotations` styles and
6792.md two identical centred table-cell styles.
Co-Authored-By: Claude <noreply@anthropic.com>
The abstract was wrapped in a Div with `custom-style` set to
`Abstract`, but then handed to `metaToContext`, which renders metadata
fields with the inline writer. That dropped the Div, and with it the
style, and also the `text:p` wrapper, leaving bare character data as a
direct child of `office:text`, where it is not valid. A
multi-paragraph abstract was collapsed into one paragraph with a
`text:line-break`.
Render the abstract separately with the block writer and remove it
from the metadata that goes through the inline writer.
This fixes the regression from 013351f602, which assumed the abstract
was filtered out of `meta'`; it was in fact set there.
Co-Authored-By: Claude <noreply@anthropic.com>
`withParagraphStyle` only wrapped `Para` in a paragraph with the
requested style; a `Plain` fell through to `blockToOpenDocument` and
came out with the default style. So, for instance, the body of a
caption-less figure, which is a `Plain`, got `Text_20_body` rather
than `Figure`.
The figure writer worked around this for the captioned case by
rewriting `Plain` to `Para` itself; that is no longer needed.
Co-Authored-By: Claude <noreply@anthropic.com>
Column style names came from `map chr [65..]`, so a table with more
than 26 columns named them `Table1.[`, `Table1.\`, and so on. A
`style:name` is an XML NCName, which those are not. Use spreadsheet
style names instead: A, B, ..., Z, AA, AB, ...
Co-Authored-By: Claude <noreply@anthropic.com>
The note counter was read before the note's body was rendered and
only bumped afterwards, so a note nested inside another one was
numbered as if the outer note did not exist and the two shared a
`text:id` (ftn0, ftn0, ftn2). Bump the counter first.
Co-Authored-By: Claude <noreply@anthropic.com>
The bookmark for a section's id surrounded the whole section, at body
level, so a screen reader met it at the section's last line rather than
at the heading. The docx reader looks for bookmarks only inside
paragraphs, so it never read that one back: the heading got an id from
its text, and links to it kept the bookmark's name, which no longer
matches since bookmark names start with an underscore. Now the bookmark
surrounds the heading's text inside the heading's paragraph, as Word
writes one, and links to headings survive a round trip.
Closes#11845. See also #8825.
Co-Authored-By: Claude <noreply@anthropic.com>
Some of the writers (gfm, docbook, asciidoc, rst) check for the
admonition name as the first class, so the change putting `alert`
first broke the output. (Regression from 3.12.)
Closes#11919.
Using "id" causes problems in some cases, because of overlap with
the id part of pandoc attributes.
Note that the Docx writer will still interpret an "id" attribute for
legacy compatibility, so if you use markdown files that specify "id",
they should still work.
Inline elements with a content body (emph, strong, underline, strike,
sub, super, smallcaps, highlight, link, lower, upper, ...) failed with
a parse error when the body contained a paragraph break or other block
content, since inline handlers parse their bodies with pInlines, which
cannot consume paragraph breaks. Example:
```
#emph[hi
there]
```
Generalize `fixNesting`, which already hoisted single block children out
of inline elements (#11017): the content of an inline element is now
split at paragraph breaks and at block children; inline runs are
wrapped back in the element, and block children are pulled out with
the styling applied to their own contents when they have a body. A
pandoc inline cannot span paragraphs, so this is the closest
structural rendering; the paragraph structure is preserved.
Remove the kludge we used for `#quote`, using `getInlineBody`; it
is no longer needed with this general fix.
A link with a non-empty title now gets a `w:tooltip` attribute on its
`w:hyperlink`, which Word shows as a ScreenTip and the docx reader
reads back as the title.
Closes#11869.
Co-Authored-By: Claude <noreply@anthropic.com>
...when parsing a block quote or list item. Otherwise
it can happen that by the time parseWithString' is called,
the position has already been set to the next file on the
command line.
Closes#11888.
Closes#11888.
Instead, generate them manually as we do for floating tables.
This removes our dependency on footnotehyper, and solves a
compatibility problem with `endfloat`.
Closes#11857.
The overall table width used `round` while individual column widths
(in colSpecListToHtml) used `truncate`, so the table width could
exceed the sum of the column widths. Update test goldens.
Co-Authored-By: Claude <noreply@anthropic.com>
Remove old ad hoc `paraToPlain` at the table cell level.
This should ensure that we don't get tables that mix
Plain and Para. Such tables tend to look funny when
rendered in docx and other formats.
Closes#11864.
With raw_html disabled (the default for the HTML reader), an inline
<style> element is now ignored rather than passed through as a
RawInline; the raw_html-enabled behavior is still tested.
Co-Authored-By: Claude <noreply@anthropic.com>
The `opt` parsers in LaTeX.hs and Citation.hs parsed the bracketed
tokens in a separate runParserT call, discarding any state changes
(macro definitions, labels, notes) made while parsing them. Since
an optional argument is not a TeX group, such changes should persist.
Use parseFromToks, which runs the sub-parse in the same parser run.
This also removes a per-optional-argument state copy.
Co-Authored-By: Claude <noreply@anthropic.com>
Commit 47610fe3d (#10781) disabled the `--`, `---`, and `"` ligatures
inside `\texttt`, but single `` ` `` and `'` characters were still
converted to curly quotes, so `\texttt{it's}` produced code containing
a Unicode right single quotation mark. Gate those conversions on the
ligatures flag too, so quote characters inside `\texttt` stay as
typed. Update test/command/3958.md accordingly.
Co-Authored-By: Claude <noreply@anthropic.com>
`tokenize pos` already produces tokens positioned relative to pos,
but the result was then shifted again by pos's line and column,
roughly doubling the reported positions of tokens retokenized from
comments in URL arguments. Tokenize from the position just after
the '%' instead, and drop the extra shift.
Co-Authored-By: Claude <noreply@anthropic.com>
