Commit Graph
1407 Commits
Author SHA1 Message Date
John MacFarlaneandClaude Opus 5.5 556a25b408 RST reader: strip minimum indentation from indented blocks.
Previously the indentation of the first line was stripped, so lines
with less indentation lost all their leading whitespace.  Now, as in
docutils, the minimum indentation is stripped and relative indentation
is preserved.

Closes #12071.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 18:30:31 -08:00
John MacFarlaneandClaude Opus 5.5 eab35c876e LaTeX reader: handle pandoc's deferred footnotes.
+ Parse `\pandocfootnotemark{n}` and `\pandocfootnotetext{n}{..}`,
  as produced by the LaTeX writer for notes in tables, captions,
  etc. Marks and texts are matched by id.
+ Resolve footnote marks inside resolved footnote texts, so that
  nested notes are not lost.
+ Track `\addtocounter{footnote}{n}`, so that `\footnotemark` and
  `\footnotetext` with counter adjustments (as produced by earlier
  versions of pandoc) are matched correctly. Previously all
  of the texts were assigned to the last mark.
+ Ignore definitions of `\pandocbounded`, `\pandocfootnotemark`,
  and `\pandocfootnotetext` (e.g. from pandoc's template), which
  the reader handles directly. Previously these definitions were
  expanded, producing garbage and losing content.

See #11936.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 20:13:38 -08:00
John MacFarlaneandClaude Opus 5.5 5bca190d12 LaTeX writer: fix hyperlinks for deferred footnotes.
Notes in tables, captions, and definition list terms are rendered
as a mark followed later by the note text. Previously we used
`\footnotemark{}` and `\footnotetext` with manual adjustments to
the footnote counter. But hyperref anchors each `\footnotetext` at
the destination of the most recent `\footnotemark`, so when there
were several such notes, all of the texts got the last mark's
anchor and the links for the other marks were broken (visibly so
with xelatex; pdflatex and lualatex silently substituted
page-level destinations).

We now emit `\pandocfootnotemark{n}` and `\pandocfootnotetext{n}{..}`,
where `n` is a unique id. These macros, defined in the default
template when the `deferred-notes` variable is set, save the
footnote mark and hyperref anchor at the mark and restore them
at the text. This removes the need for counter arithmetic, works
when the footnote counter is reset by chapter, and handles notes
nested inside deferred notes (whose texts are now emitted after
the text containing their marks). Since `\@footnotetext` is used
directly, verbatim content in such notes (`\VerbatimFootnotes`)
still works.

Users of custom LaTeX templates, or of LaTeX fragments produced
without `--standalone`, will need to include the definitions of
`\pandocfootnotemark` and `\pandocfootnotetext` from the default
template.

Closes #11936.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 20:13:38 -08:00
John MacFarlaneandClaude Opus 5.5 365cac08e7 HTML reader: don't drop ordered lists and rules inside footnotes.
Previously an `<ol>` inside the body of a note was treated as
the list of notes, and its items were lost.  Similarly, an `<hr>`
inside a note was dropped.  We now set a local `inNoteBody` flag
while parsing a note's content, and only treat `<ol>` as the
note list (and drop `<hr>`) when not in a note body.

Closes #11852.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:06:32 -08:00
John MacFarlane 160de94937 Add missing command test file to pandoc.cabal.
Also, shrink test/command/9222.docx.
2026-11-07 20:30:29 -08:00
Caio Vinícius 5f33447c92 Docx reader: follow w:numStyleLink in abstract numberings (#11978)
Word can define a list through a numbering style: the abstractNum
referenced by the list has no `w:lvl` of its own, only a
`w:numStyleLink`, and the levels live in the abstractNum whose
`w:styleLink` names the same style (ECMA-376 17.9.21 and 17.9.27).
The reader only looked at the referenced abstractNum, found no levels,
and turned the list items into plain paragraphs.

When an abstractNum has no levels but has a `w:numStyleLink`, take the
levels of the abstractNum with the matching `w:styleLink`.

Closes #9222.
2026-11-07 20:08:26 -08:00
Patrick CreightonandCopilot 5579187cdf Docx reader: read all underline types. (#11995)
Only `w:u` with `w:val="single"` was read as Underline. Double, thick,
dotted, wavy and the other underline types, which Word also shows as
underlined text, were dropped. Every value except `none` now gives
Underline, as `w:highlight` already does for highlighting.

Closes #11988.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-11-07 20:02:29 -08:00
John MacFarlaneandClaude Opus 5.5 e39194e939 ODT reader: support the styles extension. (#12065)
With `+styles`, named paragraph and character styles are recorded
in `custom-style` attributes on Divs and Spans, as in the docx
reader. Automatic styles are resolved to the named style they are
based on, and the user-visible name of the style is used.
The default paragraph style, default table cell styles, and styles
that pandoc interprets itself (Emphasis, Strong Emphasis, Source Text,
links, table captions) are not recorded. Paragraphs in list items
and image captions are not wrapped. Headings get a `custom-style`
attribute if their style is not the standard one for their level.

OpenDocument writer: encode custom style names as LibreOffice does,
e.g. `Text body` becomes `Text_20_body`, so that they match
the styles in a reference doc.

See #12060.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 19:53:49 -08:00
Václav Haisman 660898056c LaTeX: Allow repeated longtable captions. (#4623)
Add a `repeat-caption` table class that repeats captions on each page of a
longtable without adding duplicate list-of-tables entries. Preserve the
existing non-repeating default and ignore the class for floating tables.

Closes #4558.
2026-11-07 09:39:35 -08:00
John MacFarlane 8b5af1d44e Minimize test docx, odt, pptx files.
This reduces the size of the test files by 1.79 MB.
2026-11-06 10:49:26 -08:00
John MacFarlane 6e3335aad8 RST reader: require descriptions in option list items.
Closes #12063.
2026-11-06 08:19:20 -08:00
John MacFarlane d13c664ffd Citeproc: removeFinalPeriod - act across Link boundary. 2026-11-05 23:34:27 -08:00
John MacFarlane f66b168159 Depend on dev citeproc. 2026-11-05 22:49:23 -08:00
Asuka 4ddc58eeaf LaTeX reader: fix duplicated text after a control space at line end (#12055)
The tokenizer represents a backslash followed by a newline as a
`CtrlSeq " "` token whose text contains the newline.  Its end
position was advanced with `incSourceColumn`, which does not reset
the column when the text contains a newline.  As a result,
`rawLaTeXParser` computed an end position on the wrong line, and
raw LaTeX inline or block content containing such a line break
was parsed a second time after the raw span.  Use a position
advance that accounts for newlines.

Closes #12051.
2026-11-04 19:52:32 -08:00
John MacFarlane 42c5ba2038 Docx writer: implement flip-flopping for emph.
Emph within emph becomes upright.  This is important for correct
formatting of titles in CSL, among other things.  LaTeX does this
automatically, and in HTML we achieve it via CSS.

Closes #11961.
2026-11-01 09:10:12 -08:00
Caio Vinícius 6cb2494cb8 Docx reader: merge adjacent styled code blocks (#11977)
Without `+styles`, consecutive "Source Code" paragraphs are combined
into a single code block by `combineBlocks`. With `+styles`, each
paragraph is first wrapped in a `Div` carrying its `custom-style`
attribute, so the `CodeBlock` case no longer matched and every line
became its own `Div [CodeBlock]`.

Add a case that merges adjacent `Div [CodeBlock]` blocks when both the
`Div` and the `CodeBlock` attributes are equal.

Closes #9183.
2026-10-29 12:43:19 -07:00
Jeremy Levartovsky 8605c4aa6e ipynb: make ANSI escape stripping linear. (#12007)
filterIpynbOutput stripped ANSI escapes from output code blocks by
rebuilding the text one character at a time with T.cons, so time was
quadratic in the length of the output. A notebook with one 32,000-line
stream output took 4.3 s to convert to HTML; the notebooks attached to
#7744 took 42 s. Split at each ESC [ with T.breakOn and join the pieces
once instead. Output is unchanged.

Add test/command/7744.md, since nothing tested the stripping itself.

Closes #7744.
2026-10-28 11:46:04 -07:00
Alain LesageandAlain Lesage 60725b8673 EPUB writer: skip dates that cannot be normalized. (#12002)
Such dates produced an empty dc:date, which epubcheck rejects.
They are now dropped; if no date remains, the current time is used,
as when no date is given.

Co-authored-by: Alain Lesage <15161099+Alain-L@users.noreply.github.com>
2026-10-27 20:20:02 -07:00
Alain LesageandAlain Lesage d4c1c4784f EPUB writer: fix id clash with added title header. (#12003)
When content precedes the first level-1 header, a header with the
title is added. Its generated identifier could duplicate an existing
header's, so the TOC and internal links pointed to the wrong chapter.

Co-authored-by: Alain Lesage <15161099+Alain-L@users.noreply.github.com>
2026-10-27 20:18:05 -07:00
John MacFarlaneandClaude Opus 5.5 dd7fc4a0be Reset sourcepos when parsing footnotes and grid table cells.
Extends the fix for #11888 (c8c68ec0a) to footnote definitions in
the Markdown reader and to grid table cells.  Otherwise, when one of
these ends a file, relative paths in it can be rebased against the
next file on the command line.

Closes #12001.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 20:14:24 -07:00
Patrick CreightonandCopilot a5af15c2d5 Docx reader: fix namespaces in notes and comments. (#11996)
The contents of footnotes, endnotes and comments were read with the
namespace prefixes declared in word/document.xml rather than in their
own parts. If word/footnotes.xml, word/endnotes.xml or word/comments.xml
declared a prefix that document.xml did not, such as `r` or `m`, the
elements using it were dropped: a hyperlink in a footnote lost its text,
and an equation in an endnote or comment disappeared. They are now read
with the namespace maps that archiveToNotes and archiveToComments
already compute.

Closes #11991.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-24 17:33:07 -07:00
John MacFarlane 2512e1b6e7 Markdown writer: fix regression with code block language.
We try to identify which class is a language name and emit
that as the gfm style info string.

If none of the classes match a recognized language, before
3.10.1 we emitted the first class; fater 3.10.1, we emit
nothing. This commit reverts to the pre-3.10.1 behavior.

Closes #11987.
2026-10-24 16:28:23 -07:00
Patrick CreightonandCopilot b7019a9fb0 Docx reader: keep grid columns that have no width. (#11997)
`w:w` is optional on `w:gridCol`, but elemToTblGrid skipped the grid
columns that had none. The table then had fewer columns than its rows
had cells, and the extra cells were dropped. Every grid column is now
kept. If any of them has no (readable) width, all columns get the
default width, as for a table without `w:tblGrid`, since writers don't
handle a mix of specified and default widths well.

Closes #11989.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-24 11:04:52 -07:00
Aslak HellesøyandClaude Opus 5.5 f4d4af66db Docx reader: read w:fldSimple fields. (#11946)
A `w:fldSimple` element had no case in `elemToParPart'`, so `mapD` dropped it
together with its result runs: "Figure <fldSimple>1</fldSimple> Caption"
read as "Figure Caption", and a simple REF field lost its text. It is now
read as a Field with its runs as children, like a complex field.

`stripCaptionLabel` relied on that drop to remove the number after an
English Figure/Table label; it now drops the number field explicitly, so
those captions read as before.

Closes #11944.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 09:00:13 -07:00
John MacFarlane c08b112ffa Fix command test 9197.
We need an explicit `-o -` to get docx to go to stdout.
2026-10-22 19:45:47 -07:00
John MacFarlaneandClaude Opus 5.5 979c7a9779 Docx writer/reader: fix multi-paragraph list items.
Previously the docx writer gave the second and subsequent paragraphs
of a list item numbering from a separate list with an invisible
bullet.  This looked right, but semantically these paragraphs were
items of another list, which is what Word, screen readers, and
pandoc's own docx reader saw; the list was also split in three.

The writer now gives these paragraphs no numbering, just an
indentation that lines them up with the text of the list item
(as Word's "List Continue" styles do).

The docx reader now records the indentation of each list level
and tracks the open list levels; an unnumbered paragraph whose
left indentation matches the text indentation of an open level
is treated as a continuation of that level's current item,
rather than as a block quote.  So multi-paragraph list items
now round-trip through docx.

Closes #9197.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 23:11:04 -07:00
John MacFarlaneandClaude Opus 5.5 42626eef1d LaTeX reader: detect macro expansion loops across doMacros calls.
A macro whose expansion contains itself after an unknown command,
e.g. `\def\small{\@setfontsize\small\@ixpt{10}}`, was expanded
again and again at the same position, each time in a fresh call
to doMacros, so the existing depth check never triggered and
pandoc hung.  We now track consecutive expansions of the same
macro at the same position and raise PandocMacroLoop after 1000.

Closes #11088.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:12:31 -07:00
John MacFarlane d8d35b59ce RTF reader: fix tokenization bug.
We were consuming certain symbols after `\uNNNN` during tokenization.
This led to an *extra* character being consumed as the "fallback,"
so letters went missing after encoded unicode characters.

Closes #11981.
2026-10-21 12:04:08 -07:00
John MacFarlane c91ba8bc0a Typst writer: escape ... and -? ligatures.
Closes #11979.
2026-10-21 00:01:43 -07:00
John MacFarlane 0c2e12283f Use dev version of citeproc 2026-10-18 09:38:41 -07:00
John MacFarlaneandClaude Opus 5.5 1d49297127 Markdown, RST, Haddock writers: don't drop notes nested in notes.
Rendering a note can add further notes to the state. Previously
these were lost. Now we keep rendering notes until no new ones
appear, as the Org writer already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:30:28 -07:00
John MacFarlane 044472d4d9 Use yamlet for YAML parsing.
Pandoc parsed YAML with the yaml package, which depends on the C
library libyaml. It now uses yamlet, a pure Haskell YAML 1.2 parser.
Values still go through the existing aeson instances.

Behavior changes:

- YAML errors show the file, the line and the column, with the
  offending line. In metadata blocks the line is the line in the
  document.
- `yes`, `no`, `on`, `off`, `y` and `n` are strings,
  not booleans, as in YAML 1.2. This restores the behavior before
  the switch from HsYAML.
- Duplicate keys in a mapping are an error. They were a warning.
- .5, .inf and .nan are numbers.

Text.Pandoc.Translations: readTranslations takes the path of the
file, for error messages [API change].
2026-10-13 08:59:17 -07:00
John MacFarlane eafe417098 LaTeX writer: use braces around term containing =.
This is needed for the new LaTeX list code, which is triggered
by using `pdfstandard`.

See https://github.com/latex3/tagging-project/issues/1039.

Closes #11954.
2026-10-12 19:00:52 -07:00
Aslak Hellesøy 763c555a05 Markdown writer: write wide tables as grid tables.
A table was written as a grid table only if it had spans or a footer,
or if `writerColumns >= 8 * numcols`.  A wider table with block content
in a cell therefore fell back to raw HTML, even with grid_tables
enabled; with the default 72 columns that is any table of ten or more
columns.

The width check was added in ea619bfcb (2017, #2834) to fix grid
tables nested in grid tables.  The old grid writer gave each column
`floor(columns * share) - 3` characters and rendered cell contents with
`writerColumns` reduced to the column width, so nested tables got
narrower at each level.  Below about 8 characters per column those
widths reached zero or went negative and the layout broke.

The `gridTable` rewrite in 01257ef6e (pandoc 3.7) removed that failure
mode.  A column is never made narrower than its content (or, when
wrapping, than its longest unbreakable word), and cells are rendered
with unchanged `opts`, so `writerColumns` no longer shrinks for nested
content.  The RST and Muse writers already call the same `gridTable`
with no width check, and the Markdown writer already writes grid tables
wider than `columns` when a table has spans or a footer.

The check is therefore removed.  This follows the preference stated in
7c2dd0359 for markdown tables over raw HTML.  No existing test output
changes.

Closes #11945.
2026-10-10 10:46:25 -07:00
John MacFarlane 649aae2f4c SelfContained: fix regression in escaping of data URIs.
A change in 3.12 broke escaping of data URIs, so that
`--embed-resources` no longer worked properly with JavaScript
includes, breaking slide show formats.

Closes #11942.
2026-10-07 09:03:28 -07:00
John MacFarlane de45c51eb9 Fix test for #11939. 2026-10-07 08:35:33 -07:00
John MacFarlane eaf3404795 Markdown writer: Don't use a multiline table for a header-only table.
(It won't read back as a table.)

Closes #11939.
2026-10-06 23:30:10 -07:00
Erik Demaine c248f880e5 Math: respect TeX groups and comments. (#11887)
Prevent math delimiters inside TeX brace groups and percent comments
from prematurely closing an equation:

* Generalize the existing inline `\text` group handling to arbitrary
  brace groups, and support groups in display math too.
* Recognize unescaped `%` as starting a comment through the next newline.
  Ignore any braces or math delimiters within it.

Developed with OpenAI Codex.

PR revised by John MacFarlane.
2026-10-05 15:34:23 -07:00
John MacFarlane f98e0d21d7 Test the fodt --link-images reference in the Haskell test suite.
The command test extracted the xlink:href by piping pandoc's output
into grep, and the command tests run under cmd.exe on Windows, where
there is no grep and the single quotes around the pattern are not
stripped.  Nothing matched, and the test failed.

Tests.Writers.ODT makes the same assertion by parsing the output and
comparing the xlink:href of every draw:image.  Reading the output back
in would not do instead: fixRelativeLink strips a leading "../", so a
flat file that wrongly had one would still read back as "lalune.jpg".
2026-10-04 01:44:27 +00:00
John MacFarlane 7b400e6229 Don't extract particles from given and family names...
in structured CSL YAML references.  Thus,

    family: de Gaulle

should not create a non-dropping particle "de"; "de Gaulle"
should be considered the integral family name.

This is a behavior change that could affect some bibliography
processing.

The commit 4be49d0ce9 is
also reverted (adding a section explaining the former behavior
to the manual).

Closes #11911.
2026-10-03 18:03:12 -07:00
John MacFarlane 58ec1c976d Add tests for the fodt reader and writer.
The fixtures in test/odt/fodt are LibreOffice's own conversions of the
zipped fixtures in test/odt/odt, and they are compared against the very
same expectations, so that flat and zipped input are asserted to
converge on identical documents.

There are only four of them, because a flat file shares nearly all of
the reader with a zipped one: once readFODT has handed the root element
over, the converter is the very same code.  Two of the four are there
for the two paths that a flat file alone reaches: image, whose data has
to be decoded from office:binary-data and typed from draw:mime-type
rather than found in the archive, and formula, whose MathML is read
from the draw:object itself rather than from a sub-document.  The other
two guard against whitespace, which is peculiar to the format without
being a separate branch of the reader, since LibreOffice pretty-prints
a flat file and that indentation is in the XML the reader sees:
preformattedText is a CodeBlock, whose whitespace is significant and so
is compared exactly, and textMixedStyles is a run of nested emphasis,
where stray indentation would show up between the inlines.

`test/command/fodt-roundtrip.md` round-trips markdown through the flat
format, which also covers the writer, whose output has a timestamp in
it and so cannot be compared byte for byte.

Together with previous commits, closes #4010.
2026-10-03 14:47:02 -07:00
John MacFarlane 8f949dbb09 Revert "Typst writer: omit blank line at end of block."
This reverts commit 6accb0b9f7.
2026-10-02 21:43:46 -07:00
John MacFarlane 8d2e8435a5 Fix escaping bug in commonmark writer.
A non-alphanumeric after an escaped backslash would be omitted.

Closes #11927.
2026-10-02 10:10:36 -07:00
John MacFarlane 2d244c7180 LaTeX writer: allow alt text in figure to wrap.
Closes #11924.
2026-10-02 10:00:31 -07:00
John MacFarlaneandClaude 4580225a02 OpenDocument writer: sort automatic text styles numerically.
The text styles were kept in a map keyed on the set of style
attributes, with the generated style name as part of the value, and
that name was used to order the style definitions in
`office:automatic-styles`.  Sorting names as text puts `T10` between
`T1` and `T2`, so a document with ten or more distinct text styles
emitted them in a confusing order.

Store the style's number rather than its name, which makes the
existing sort numeric, and derive the name from it.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-01 05:27:32 +00:00
John MacFarlaneandClaude 008afcaba1 OpenDocument writer: reuse identical automatic paragraph styles.
Every blockquote, every explicitly aligned table cell, and every
direction-adjusted style got a fresh `Pn` automatic style, even when
an identical one already existed. A document with many of them paid
for a style definition each time, and the style list that gets walked
when emitting `office:automatic-styles` grew with it.

Key the styles on their attributes and paragraph properties and reuse
a match. The direction cache added for RTL support is subsumed by
this one and goes away.

The test output changes only by dropping duplicate definitions:
`writer.opendocument` had three identical `Quotations` styles and
6792.md two identical centred table-cell styles.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-01 05:18:49 +00:00
John MacFarlaneandClaude 8f46b9a6c9 OpenDocument writer: render the abstract as blocks.
The abstract was wrapped in a Div with `custom-style` set to
`Abstract`, but then handed to `metaToContext`, which renders metadata
fields with the inline writer. That dropped the Div, and with it the
style, and also the `text:p` wrapper, leaving bare character data as a
direct child of `office:text`, where it is not valid. A
multi-paragraph abstract was collapsed into one paragraph with a
`text:line-break`.

Render the abstract separately with the block writer and remove it
from the metadata that goes through the inline writer.

This fixes the regression from 013351f602, which assumed the abstract
was filtered out of `meta'`; it was in fact set there.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-01 04:36:19 +00:00
John MacFarlaneandClaude 8a58b75b62 OpenDocument writer: apply the requested style to Plain blocks.
`withParagraphStyle` only wrapped `Para` in a paragraph with the
requested style; a `Plain` fell through to `blockToOpenDocument` and
came out with the default style. So, for instance, the body of a
caption-less figure, which is a `Plain`, got `Text_20_body` rather
than `Figure`.

The figure writer worked around this for the captioned case by
rewriting `Plain` to `Para` itself; that is no longer needed.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-01 04:34:53 +00:00
John MacFarlaneandClaude dcb1adde39 OpenDocument writer: use valid column style names past column 26.
Column style names came from `map chr [65..]`, so a table with more
than 26 columns named them `Table1.[`, `Table1.\`, and so on. A
`style:name` is an XML NCName, which those are not. Use spreadsheet
style names instead: A, B, ..., Z, AA, AB, ...

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-01 04:33:44 +00:00
John MacFarlaneandClaude 0cb6ff4fe6 OpenDocument writer: give nested notes distinct ids.
The note counter was read before the note's body was rendered and
only bumped afterwards, so a note nested inside another one was
numbered as if the outer note did not exist and the two shared a
`text:id` (ftn0, ftn0, ftn2). Bump the counter first.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-01 04:32:18 +00:00