Commit Graph
9472 Commits
Author SHA1 Message Date
John MacFarlane ceb0917599 Ensure that IO error messages go to stderr.
Closes #11710.
2026-07-01 13:09:47 +02:00
John MacFarlane 8547196b44 Markdown reader: ordered sublists must start with 1
(or equivalent, e.g. 'i').  This brings the markdown
parser in line with commonmark, and avoids a lot of
unintended list starts.  Closes #11735.

Note that ordered lists that are not sublists may
still start with numbers other than 1.
2026-07-01 11:17:21 +02:00
John MacFarlane 76bd78f653 CommandLineOptions: use optCSL and optBibliography...
from Opt.  Previously we set optMetadata directly, but that
had some unintended effects, especially in interacting with
defaults files. With this change, a `--csl` on the command
line can override one set in a prior defaults file.

Closes #11741.
2026-06-30 15:28:05 +02:00
John MacFarlane 612e143fbe DokuWiki reader: skip non-cell content after table row.
Closes #11739.
2026-06-29 19:28:54 +02:00
John MacFarlane 1c22498269 LaTeX writer: fix footnotes in def list headings.
Closes #8240.
2026-06-29 11:40:30 +02:00
John MacFarlane 197ea8e8bd RST reader: treat undefined substitution references as plain text.
This mirrors rst2html behavior. Closes #11727.
2026-06-23 12:06:27 +02:00
John MacFarlane 58929eb70c Markdown writer: in YAML metadata, put double quotes around...
...strings that are numbers beginning with 0 or ending with 0 and
having a decimal point.  Otherwise they will read as YAML numbers
and potentially be modified (e.g. 3.10 -> 3.1).

It would be better to fix this on the reader side, but since
we use a standard YAML parser it's hard to see how.

Closes #11715.
2026-06-21 15:15:42 +02:00
John MacFarlane da82107116 Change adding of .yaml extension in --defaults option.
Previously it was always added if the specified filename
lacked an extension. Now it is only added if there is
no file with the original name.  So, these are tried in
order: `FILE`, `FILE.yaml`, `DATADIR/defaults/FILE`,
`DATADIR/defaults/FILE.yaml`.

This facilitates using `--defaults` with bash/zsh `<(..)` or
`=(..)`.

Closes #11717.
2026-06-18 23:27:17 +03:00
John MacFarlaneandClaude 850c7b9613 App: enable auto_identifiers when a table of contents is requested.
Table-of-contents entries link to heading identifiers, so when
`--toc/--table-of-contents` is given, force the `auto_identifiers`
extension on for the input format even when it is off by default
(as it now is for typst).  This makes typst-to-markdown `--toc`
produce working links.

Closes #11041.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 23:40:17 +02:00
John MacFarlaneandClaude ec4fb91049 Typst reader: support auto_identifiers extension.
Headings without an explicit label can now be assigned automatic
identifiers based on the heading text, making them linkable in a
generated table of contents.  The extension is available for the
typst reader but is off by default; enable it with
`-f typst+auto_identifiers`.  The related `gfm_auto_identifiers` and
`ascii_identifiers` extensions are also made available.

Closes #11041.

Text.Pandoc.Readers.Typst.Parsing: PState gains sOptions,
sIdentifiers, and sLogMessages fields, and now has
HasReaderOptions, HasIdentifierList, and HasLogMessages
instances, allowing reuse of the shared registerHeader. (Not an
API change.)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 23:38:51 +02:00
Robertas 853ae7fecc HTML reader: fix unused variable warning in CI (#11704) 2026-06-13 10:00:18 +02:00
Robertas f751000aa4 Docx writer: fixes for test failures in #11697 (#11703) 2026-06-12 21:07:59 +02:00
John MacFarlane 8a52ac1493 Markdown writer: improve identification of code language class...
...by looking it up in the syntax map. (Previously we just used
the first, excepting `sourceCode`.)

Closes #11701 (together with previous commit).
2026-06-12 21:01:25 +02:00
John MacFarlane 6049ab9c09 HTML reader: strip language- from all classes, not just first.
Also combine classes from both code and pre class if they are
nested.

See #11701. Improves on the fix to #7221.
2026-06-12 21:00:59 +02:00
Robertas 06af643730 Docx writer: use namespace URI instead of prefix in extractPageLayout (#11697)
When a reference document uses a non-standard namespace prefix for the
WordprocessingML namespace (e.g. `ns0` instead of `w`), `sectPr` elements
copied from the reference would retain the non-`w` prefix, producing
malformed XML in the output document.  Similarly,
`extractPageLayout` only matched elements with prefix `w`, missing
`sectPr` elements with other prefixes.  This is fixed by matching on the
namespace URI rather than the prefix, and normalizing the prefix to `w`
on all elements and attributes copied from reference-doc `sectPr`.

Some new tests have been added, and the test suite has been streamlined
using helper functions.
2026-06-12 18:13:10 +02:00
John MacFarlane 912bfa5e2e Use starmath for math in ODT.
This improves appearance and editability of math in ODT.

Depends on dev version of texmath.

The main code changes are due to @jdpipe (#11470).
2026-06-09 13:06:17 +02:00
John MacFarlaneandClaude 1b4a5356b4 Docx reader: key list numbering off abstractNumId.
Word represents "restart numbering" on a style-based list by pointing
only the first item of the restarted list at a new `numId` that shares the
original list's abstract numbering definition but carries a
`w:startOverride`; the remaining items keep using the original `numId`.

Pandoc keyed list continuation and grouping on the `numId`, so the
restarted items continued the stale count from the earlier list (and
were split into a separate ordered list with the wrong start).

Key continuation and grouping off the `abstractNumId` instead (the real
running counter in Word), and treat `startOverride` as a restart that
resets the count.

Closes #8367.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-09 11:59:10 +02:00
John MacFarlane 17eb8e4f8b ODT reader: Remove an unnecessary definition. 2026-06-09 10:04:53 +02:00
John MacFarlane 32d66351a6 ODT reader: don't create Emph just because of changed font.
Closes #11691.
2026-06-09 07:26:56 +00:00
John MacFarlaneandClaude 3a5f7ecee1 RTF reader: support legacy \pn/\pntext paragraph numbering (#11686).
Word-95-style numbered and bulleted lists are encoded with a
`{\pntext ...}` auto-number destination at the start of each list
paragraph plus a `{\*\pn ...}` destination describing the numbering,
rather than the modern `\listtext`/`\listtable` mechanism.  Two problems:

1. The `\pntext` marker text ("1.", "·", etc.) was captured as the
   paragraph's first text run, which sits before the paragraph's
   `\ls`/`\ilvl`, so emitBlocks (which reads list properties from the first
   run) misclassified the paragraph as an ordinary paragraph.  The first
   item of each list therefore came out as a stray paragraph.

2. The numbering style was ignored, so numbered lists defaulted to
   bullets.

Treat `\pntext` like `\listtext`: drop its visible marker text and flag the
start of a new list item.  Parse the `{\*\pn ...}` destination
(`\pnlvlbody`/`\pnlvlblt`, `\pndec`, `\pnucltr`, `\pnlcltr`,
`\pnucrm`, `\pnlcrm`, `\pnstart`, and the `\ls`/`\ilvl` keys it
carries) into the list override table so numbered lists are
emitted as ordered lists with the right number style.

`\pn` is a paragraph property that remains in effect until reset
by `\pard`, auto-numbering every paragraph in scope. Track
this (`sPnActive`) so each paragraph becomes its own list item,
rather than merging markerless continuation paragraphs as is done
for modern `\listtext` lists.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-09 08:49:17 +02:00
John MacFarlaneandClaude 656a6704a5 RTF writer: emit real RTF lists with a list table.
Lists were written as plain paragraphs with a literal marker and a
hanging indent, carrying none of the `\ls`/`\ilvl` references or the
`\listtable`/`\listoverridetable` that the RTF list model (and pandoc's
own reader) expect, so lists could not round-trip.

Walk the document tagging each list with a unique id and nesting level
(`prepareLists`), build a `\listtable`/`\listoverridetable` describing
each list (`listTableRTF`, exposed via the `listtable` template
variable), and render every list paragraph with its `\ls`/`\ilvl`
reference.  The first paragraph of each item gets a `{\listtext}`
marker; continuation paragraphs keep the reference but omit it, so
multi-paragraph items round-trip.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-09 00:17:57 +02:00
John MacFarlaneandClaude 9312f1da1b RTF reader: support multi-paragraph list items.
The reader treated every list paragraph as a new list item, so a list
item containing several paragraphs was split into one item per
paragraph.  In the RTF list model a new item is marked by a
`{\listtext ...}` destination group; a list paragraph lacking one is a
continuation of the current item.

Track whether a `{\listtext}` group was seen (`sListText`) and, in
`emitBlocks`, append a continuation paragraph (no `\listtext`) to the
current item rather than starting a new one.

Adds a `list_multiparagraph` reader test.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-09 00:17:20 +02:00
John MacFarlane 5566a6bf3c Docx reader: improve horizontal rule detection.
Previously we failed to detect horizontal rules when the
underlying XML contained text elements for semantically
insignificant whitespace. We also failed to handle the case
where a paragraph contains text and has a bottom border
property: this seems to be what Word does when you insert
a horizontal rule, e.g. by typing a series of hyphens or #
characters.

Closes #11689.
2026-06-08 23:49:22 +02:00
John MacFarlane aafcfa14b0 LaTeX reader: handle \hypertarget with one braced parameter.
This seems to work in LaTeX, no doubt because a single token
is consumed as the target text parameter.

Closes #7418.
2026-06-08 19:22:24 +02:00
John MacFarlane 16d66edb2a Typst writer: fold lang into pickTypstAttributes.
This ensures that `lang` attributes will be set for both
Divs and Spans. Closes #11673.
2026-06-06 22:49:45 +02:00
John MacFarlane 9d68bb83a7 T.P.Writers.Shared; htmlAttrs - don't add data- prefix for markdown.
This is necessary for the `markdown_attribute` extension to
work properly. Closes #11685.
2026-06-05 17:17:22 +02:00
John MacFarlaneandClaude 098c48d79f RTF reader: fix tables parsed as deeply nested tables.
Closes #11682.

Fixes two bugs that caused a simple table to be read as a cascade of
nested tables:

* `\plain` reset the entire property record (including the in-table
  flag) via `const def`.  When a cell paragraph used `\plain` after
  `\intbl`, the in-table flag was cleared, so the next `\cell` closed
  the partially-built table and embedded it inside a new cell.  Per the
  RTF spec, `\plain` should reset only character formatting, so it now
  preserves paragraph/context properties (in-table, list level, outline
  level, hyperlink, anchor).

* `\row` was ignored and only `\trowd` started a new row.  Real-world
  RTF often emits a single `\trowd` and separates rows with `\row`, so
  every cell ended up in one row.  Both `\trowd` and `\row` now begin a
  fresh row (only when the current one has cells), and empty trailing
  rows are dropped when closing the table.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-03 16:36:29 +02:00
John MacFarlane 676ccdc904 T.P.Writers.Shared: htmlAttrs: use data- prefix when needed.
This now behaves like the HTML writer, adding the `data-`
prefix for custom attributes.  This will affect, e.g.,
the mediawiki writer.

Closes #11680.
2026-06-03 12:23:44 +02:00
John MacFarlane 1e37beac98 Remove unnecessary import. 2026-05-31 16:54:54 +02:00
John MacFarlane f0b9d7d3cc T.P.Error: display IOError without stack trace.
Closes #11676.
2026-05-31 11:30:32 +02:00
John MacFarlane 328a9fcc6e Docx writer: fix display of paragraph with just inline math.
Word displays a paragraph containing just one inline math
element as if it were display math. We don't want that,
so we add a zero-width space to defeat this behavior.

Closes #11674.
2026-05-31 10:43:39 +02:00
Meher Chaitanya 1ffeb856ee Add auto_identifiers support to Man Reader (#11675)
Add support for the `auto_identifiers`, `gfm_auto_identifiers`, and
`ascii_identifiers` extensions in the man reader. Section headings
parsed from .SH and .SS macros now receive auto-generated id
attributes when the extension is enabled, enabling `--toc` to
produce working anchor links.

- Add `autoIdExtensions` to default man extensions [behavior change]
- Add `HasReaderOptions`, `HasLogMessages` and `HasIdentifierList` to
  `ManState` to run `registerHeader`

Closes #8852.
2026-05-30 19:11:14 +02:00
Amir Dekel c6ba4d9040 Prevent wrong combination of subtitle and titleaddon with maintitle (#11677)
When `maintitle` is present, `subtitle` should not be added to `title`, but only
to `volume-title`.
2026-05-30 19:01:19 +02:00
Johan Larsson 41f829c43d Markdown reader: allow grid tables to be indented. (#11671)
Like the other table syntaxes (pipe, simple, and multiline tables) and
block-level constructs generally, a grid table may now be indented by up
to three spaces and still be recognized as a table.  Previously the
grid-table parser required the table to begin at the left margin, so an
indented grid table was parsed as a paragraph.

The leading indentation is stripped uniformly from each line before the
table is parsed, so an indented grid table produces the same AST as its
non-indented equivalent.

Adds a command test.
2026-05-28 00:07:29 +02:00
John MacFarlane 1c7cfb2a2a OpenDocument/ODT writer: use predefined styles. (#11672)
Previously the OpenDocument writer emitted a fresh automatic style
(L1..Ln, P1..Pn, T1..Tn) for nearly every list, list-item paragraph,
block quote, preformatted block, and inline text style.  This produced
large ODT files, made `--reference-doc` customization ineffective (the
user's predefined styles were never referenced), and gave each list its
own indentation independent of any containing block quote.

This commit teaches the writer to reference the predefined styles that
LibreOffice ships and that pandoc's reference.odt now exports:

- Bullet lists use `List_20_1`; ordered lists with default start and
  decimal format use `Numbering_20_1`.  Non-default ordered lists
  generate a single named override style (`Pandoc_Numbering_N`)
  memoised by (ListNumberStyle, ListNumberDelim); a non-default start
  value with the default format is expressed via `text:start-value`
  on the `text:list` element instead of a new style.
- List-item paragraphs use `List_20_Bullet[_Tight]` and
  `List_20_Number[_Tight]`.  The Tight variants are pandoc-specific
  (zero top/bottom margin) and are injected into the user's
  reference.odt if missing, just like the Skylighting token styles.
- Block quotes use the predefined `Quotations` paragraph style
  directly.  Nested block quotes use a single automatic style that
  inherits from Quotations and only adds extra margin-left, so a list
  inside a block quote now inherits its container's indent (#2747).
- Preformatted blocks use `Preformatted_20_Text` directly.
- Emphasis, Strong, Strikeout, Subscript, Superscript and Code spans
  use the predefined `Emphasis`, `Strong_20_Emphasis`, `Strikeout`,
  `Subscript`, `Superscript` and `Source_20_Text` text styles.
- `paraStyle`/`paraStyleFromParent` no longer emit a wrapper automatic
  style when its only attribute would be `parent-style-name`; the
  parent name is returned directly.

Closes #9136.
Closes #5086.
Closes #2747.
Closes #3426.
Closes #7336.

Co-authored by: Claude Opus 4.7.
2026-05-27 17:35:39 +02:00
John MacFarlane 9c83002851 Use MathJax v4 in default HTML templates.
See #11669.
2026-05-26 22:44:17 +02:00
John MacFarlane d37f306299 Man reader: better handling of .TP macro.
We parse these as DefinitionList items, but we previously
sometimes stopped prematurely in including material in the
definition.  We should include everything until we hit a new
indentation-changing macro.

Closes #11668.
2026-05-26 12:57:29 +02:00
Sai Asish Y 8922be0530 Docx writer: fix empty keywords in core document properties (#11666)
`stringify` returns the empty string for a MetaString, so each keyword
in the `cp:keywords` list of `docProps/core.xml` was rendered as empty.
Convert each metadata value like `lookupMetaString` does instead.

Signed-off-by: Sai Asish Y <say.apm35@gmail.com>
2026-05-25 15:16:18 +02:00
John MacFarlane 646eeb6614 gridTable: fix calculation of column widths for default columns.
This fixes a bug which produced too-narrow columns in some cases.

Closes #11664.
2026-05-23 18:09:46 +02:00
nibras shami 0640c4c985 EPUB writer: support multiple EPUB versions for raw content (#11628).
This change ensures that raw content marked `epub2` will appear in (only) EPUBv2 output
and content marked `epub3` will appear in (only) EPUBv3 output.
2026-05-17 11:02:34 +02:00
John MacFarlane a263eac360 Typst writer: add zero-width space before a Span label...
...if otherwise the label doesn't come after anything.
(In this case typst will raise an error.)

Closes #11568.
2026-05-15 15:25:14 +02:00
John MacFarlane 73a48d2670 HTML reader: parse aside as a Div.
(Instead of using raw HTML.)

The "aside" class is added to the Div.
Also, add "header" class to Divs created from headers.

See #11626.
2026-05-14 11:58:19 +02:00
John MacFarlane b43762cdc4 Error messages: use single quotes around paths and format names.
Closes #11645.
2026-05-13 17:15:34 +02:00
John MacFarlane 4e3707543b Docx reader: fix bug in bitmask checking.
This led to some table rows being wrongly considered header rows.

We now correctly handle the example from
https://github.com/jgm/pandoc/issues/8299#issuecomment-4397472060

See #8299.
2026-05-12 15:33:45 +02:00
John MacFarlane b754d4da68 Docx reader: improve treatment of tblHeader element.
If tblHeader exists but has `w:val="0"`, then don't consider
the element a header.

See #8299...but this change doesn't seem to fix things completely.
2026-05-12 14:59:31 +02:00
John MacFarlane 93c78e4cf6 Translations: find lang-script type translations.
E.g. for zh-Hant-TW look for (in order) zh-Hant-TW.yaml,
zh-Hant.yaml, zh.yaml.

Closes #11648.
2026-05-12 14:50:59 +02:00
John MacFarlane b8a0e5e85d Roff reader: handle \ line continuation in table cells.
Closes #11635.
2026-05-11 12:30:20 +02:00
John MacFarlane 12051ec38a Avoid trailing spaces in --help output.
Closes #11623.
2026-05-07 12:39:06 +02:00
Thomas Hodgson 5713ae7f11 Allow doi field in bibtex (input and output) (#11620)
Like `url`, this is not an official field, but it is supported by some styles,
including those used by natbib.sty.

Closes #11617.
2026-05-07 12:38:06 +02:00
John MacFarlane 7825bd001c Markdown reader: don't produce empty Raw element with --strip-comments.
Closes #11625.
2026-05-07 12:10:23 +02:00